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Executive  Summary 


This  paper  is  a  progress  report  on  research  undertaken  by  the  Institute  for  Defense  Analyses 
(IDA)  to  better  understand  Major  Defense  Acquisition  Program  (MDAP)  cycle  times — i.e.,  the 
amount  of  time  it  takes  to  develop,  produce,  and  field  a  new  weapon  system.  This  project  is  the 
second  phase  of  research  into  the  detenninants  of  MDAP  cycle  time,  including  how  program 
schedules  are  originally  set,  and  subsequently  how  they  are  managed,  to  provide  timely  capabilities 
to  Department  of  Defense  (DOD)  forces.  The  initial  phase  of  this  research,  completed  in  2014, 
reviewed  prior  research  into  issues  associated  with  defense  acquisition  cycle  times  and  defined  a 
program  of  additional  research,  focused  as  follows: 

1 .  Acquisition  schedule  development:  How  are  schedules  for  acquisition  programs 
actually  set  and  how  are  they  changed?  What  role  does  the  requirements 
community  play  in  the  process? 

2.  Management  and  oversight:  How  are  program  managers  incentivized  to  manage 
and  control  schedules?  Once  an  acquisition  program  is  established,  what  role  does 
the  requirements  community  play  in  decisions  affecting  program  schedules? 

3.  Program  definition  and  characteristics:  Which  types  of  systems  or  programs  are 
associated  with  which  problems?  Are  there  indications  of  the  root  causes  from  a 
commodity  perspective? 

This  second  phase  addressed  the  first  of  those  topics,  requirements  definition  and 
acquisition  schedule  development.  Another  related  IDA  task,  conducted  concurrently  with  this 
task,  explored  the  third  topic  by  examining  the  historical  trends  in  MDAP  cycle  times  by  program 
commodity  or  type,  and  growth  in  those  times  over  the  course  of  the  programs’  development, 
production,  and  initial  fielding.  Findings  of  that  task  will  be  drawn  upon  in  defining  the  context 
for  the  current  work. 

Approach 

To  detennine  how  schedules  are  set,  the  study  team  reviewed  both  requirements  system 
and  acquisition  system  documentation  on  the  schedule-setting  processes  and  conducted  interviews 
with  personnel  in  both  the  requirements  communities  and  acquisition  communities.  In 
coordination  with  the  sponsor,  the  team  selected  a  small  set  of  MDAPs  for  in-depth  investigation 
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of  the  drivers  that  determined  schedule  benchmarks.  The  team  sought  to  determine:  the  inputs  and 
analytical  processes  the  requirements  community  used  to  set  times  when  capabilities  are  needed 
and  how  those  parameters  are  ultimately  used  by  the  acquisition  process  to  establish  program 
schedules.  During  this  process,  the  research  team  sought  to  identify  examples  of  “best  practices” 
for  oversight  and  management  of  schedules. 

Overall  Perspectives 

The  interviews  and  document  reviews  resulted  in  the  following  overall  perspectives 
regarding  schedules  and  cycle  time: 

•  Current  requirements  documentation  does  not  facilitate  tracking  of  how  program 
schedules  are  set.  There  is  little  if  any  documentation  of  tradeoff  analyses  and 
negotiations  regarding  schedules  between  “requirers”  and  “acquirers” — 
informed  by  perspectives  on  the  maturity  of  applicable  technologies. 

•  There  is  little  evidence  that  trades  are  made  that  reduce  requirements  in  favor  of 
accelerating  schedules. 

•  Acquisition  Schedules  are  driven  by  needs  and  processes: 

o  Needs 

-  New  and  evolving  threats 

-  Obsolescence  and  associated  increased  support  costs 

-  Changes  in  priorities  or  strategy — e.g.,  shift  toward  the  Pacific 
region 

o  Processes 

-  Program/budget  factors:  execute  programs  within  available  funds 

-  Technology  maturation 

-  Industrial  base:  keep  something  in  the  pipeline  to  maintain  the 
industrial  base 

-  Acquisition  management  process  constraints 

Evaluation  of  Selected  Programs 

Based  on  guidance  from  the  sponsor,  programs  were  selected  for  in-depth  examination  that 
had  been  initiated  within  the  last  six  years  in  order  to  more  accurately  reflect  current  processes: 

•  Army:  Common  Infrared  Countermeasures  (CIRCM)  and  Integrated  Force 
Protection  Capability  (IFPC) 


IV 


•  Navy:  Next  Generation  Jammer  (NGJ)  and  Air  and  Missile  Defense  Radar 
(AMDR) 

•  Air  Force:  3-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR)  system 
and  F-22  Increment  3.2B  Modernization 

Specific  Findings 

Most  of  the  programs  examined  are  in  response  to  requirements  first  stated  in  the  2004- 
2010  timeframe  and  most  have  IOC  dates  in  the  2020+  timeframe.  In  most  of  these  programs, 
requirements  are  near-term  threat-driven,  though  obsolescence  is  also  a  factor.  It  is  our  observation 
that  for  these  programs,  the  needed  capabilities  will  be  available  several  years  after  the  threats 
emerged.  That  is,  the  threats  these  programs  are  aimed  to  address  either  exist  now  or  will  emerge 
into  operational  reality  before  the  systems  are  fielded.  That  raises  questions  of  how  threats  are 
defined  and  when  they  are  seen  as  being  realized.  From  that  we  conclude  that  for  some  programs 
there  appears  to  be  a  requirements-capabilities  mismatch,  where  the  problem  appears  to  be  that 
the  technology  needed  will  not  be  available  at  the  time  of  the  defined  threat. 

The  study  arrived  at  the  following  more  specific  findings: 

•  Requirements  inputs  on  IOC  and  other  schedule  parameters  are  difficult  to  pin  down — 
documentation  is  lacking  or  difficult  to  obtain:  For  most  current  MDAPs,  the  IOCs 
reflected  in  acquisition  documents  were  drawn  from  requirements  documents  drafted 
between  Milestones  A  and  B1 — relatively  late  in  process.  Furthermore,  it  is  not  clear 
whether  specified  IOCs  were  based  on  need  or  what  is  achievable.  Requirements 
documents  for  MDAPs  are  archived  in  the  Knowledge  Management  and  Decision  System 
maintained  on  the  SIPRNET2  by  the  Joint  Staff  (J-8).  That  system  has  proven  difficult  to 
use  and  does  not  appear  to  contain  all  relevant  documents. 

•  The  team  was  not  successful  in  obtaining  several  of  the  germinating  requirements 
documents  specific  to  systems  reviewed.  A  clear  statement  was  found  for  only  one  system 
(Air  and  Missile  Defense  Radar,  AMDR)  when  specific  threat  capabilities  were  projected 
to  be  operational. 

•  Program  schedule  setting  varies  in  rigor: 


1  Up  to  the  interim  version  of  DODI  5000.02  published  in  November  2013 — i.e.,  the  one  in  effect  when  all  the 
programs  investigated  were  initiated.  The  current  5000.02  requires  a  draft  Capabilities  Development  Document 
prior  to  Milestone  A. 

2 
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—  The  Navy  appears  to  have  a  well-defined  process  to  initiate  programs  based  on 
agreed  understanding  between  “requirers,”  developers,  and  resource  planners. 

—  The  Air  Force  has  a  similar  process  but  with  a  lesser  role  for  resource  planners. 

•  Focus  on  technology  maturation  as  part  of  early  acquisition  strategy  can  be  crucial. 

-  The  AMDR  and  NGJ  programs  showed  that  technology-driven  programs  can  be 
successful  when  attention  is  focused  on  earlier  technology  maturation.  The  CIRCM 
system  is  an  example  of  the  apparent  failure  to  do  that. 

•  Tradeoffs  of  performance,  cost,  and  schedule:  If  it  is  deemed  essential  to  meet  firm 
schedule  dates  to  support  operational  or  other  critical  needs,  it  may  be  necessary  to  make 
tradeoffs  of  perfonnance  (especially  in  regard  to  the  technologies  employed)  and/or  costs 
in  order  to  achieve  the  schedule  needed.  Although  the  current  Department  of  Defense 
Instruction  (DODI)  5000. 023  contains  several  provisions  requiring  tradeoff  analyses 
among  cost,  schedule,  and  perfonnance,  there  is  no  explicit  provision  to  consider  the 
relative  urgency  of  operational  needs  in  making  such  tradeoffs.  A  more  explicit  statement 
would  provide  greater  clarity.  Changes  may  also  be  needed  to  Joint  Staff  procedures  in 
order  to  identify  such  critical  need  dates. 

•  Funding  impacts:  It  is  difficult  to  discern  causes  from  effects  for  funding  reduction  impacts 
on  schedules. 

—  The  only  clear  case  identified  in  which  funding  issues  caused  program  delay 
was  NGJ,  which  incurred  a  double-hit  for  execution  delays  caused  by  award 
protest. 

-  Several  programs  appeared  to  lack  priority  in  the  program/budgeting  process — 
3DELRR,  NGJ,  CIRCM,  IFPC.  In  the  case  of  NGJ  and  CIRCM,  that  seems 
inconsistent  with  the  need  to  counter  near-tenn,  or  even  current,  threats. 

•  Process  delays,  such  as  protests,  acquisition  strategy  refonnulation,  and  contract  rework 
have  impacts,  measured  more  in  months  than  years,  and  they  do  not  generally  appear  to  be 
“long  poles  in  tent”  relative  to  other  factors.4 

Broader  Insights 

While  much  has  been  said  and  written  on  defense  acquisition  cycle  time,  often  these 
thoughts  do  not  include  the  obvious  question:  Why  is  there  concern  over  acquisition  cycle  time? 
Arguably,  reducing  cycle  time  per  se  should  not  be  the  objective,  since  that  may  lead  to  a  number 
of  undesired  outcomes,  such  as  greatly  increased  costs  and/or  inadequate  perfonnance.  At  times  it 

3 

Department  of  Defense  Instruction  5000.02,  “Operation  of  the  Defense  Acquisition  System,”  January  2015. 

4  An  exception  might  be  the  3DELRR  system,  the  subject  of  a  protest  and  an  associated  ongoing  lawsuit. 
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may  be  impossible  to  achieve  stated  performance  requirements  faster  given  the  limitations  of 
technology.  Trading  off  the  “required”  performance  to  get  something  out  quickly  can  have 
downsides — notably  perfonnance  that  falls  short  of  meeting  important  operational  needs.  Rather 
the  cycle  time  goal  should  be  to  structure  and  execute  acquisition  programs  to  best  meet  user  needs 
in  terms  of  both  capabilities  and  timeliness.  Though  simply  stated,  achieving  that  end  has  many 
dimensions.  It  may  not  be  technically  possible  to  provide  the  capabilities  desired  by  the  user  within 
the  desired  time  constraint — in  fact,  in  today’s  world  of  rapid  technological  dispersion,  that 
situation  might  be  the  norm.  So  tradeoffs  must  be  made  to  provide  the  best  capabilities  achievable 
by  the  time  needed  within  an  acceptable  level  of  risk. 

Thus  in  our  view,  the  real  “cycle  time”  issue  is  how  to  provide  the  operational  forces 
weapon  systems  that  have  the  needed  capabilities  and  that  are  fielded  when  needed.  To  ensure  that 
those  dual  objectives  are  met,  it  is  necessary  for  requirements  and  acquisition  communities  to  work 
together  closely  in  the  initial  stages  of  systems  acquisition  to  define  programs  based  on 
technologies  that  can  be  matured  and  implemented  in  the  required  timeframe  within  an  acceptable 
level  of  risk.  If  that  process  is  not  effectively  accomplished,  the  ensuing  program  faces  a  high  risk 
of  failure  to  achieve  its  fundamental  objective  of  providing  the  user  the  right  capabilities  at  the 
right  time. 

For  that  process  to  work  effectively  requires  rigorous  assessments  of  the  state  of 
technologies,  informed  by  technology  prototyping,  experimentation,  and  demonstration  of 
advanced  concepts,  to  assess  the  prospects  and  potential  value  to  the  user.  Also,  the  maturation  of 
underlying  enabling  technologies  needs  to  be  given  adequate  and  sufficiently  early  focus  and 
funding. 

There  is  no  “one  size  fits  all”  approach  to  achieving  the  fundamental  objective  stated  above. 
At  one  extreme,  there  are  rare  situations  where  it  is  necessary  to  seek  highly  ambitious  capabilities, 
accepting  higher  risks,  to  confront  unknowns.  Schedule  and  cost  estimates  will  be  more  uncertain. 
It  is  the  role  of  leadership  to  detennine  whether  the  requirements  for  such  capabilities  and  their 
operational  value  justify  taking  on  higher  risks.  The  risks  must  be  identified  and  articulated 
carefully  to  leadership  so  that  an  informed  decision  can  be  reached.  Moreover,  we  contend  that 
such  ambitious  and  risky  programs  should  be  given  direct  and  special  management  oversight  and 
attention  at  the  highest  levels  as  a  basis  for  their  being  undertaken. 

At  the  other  extreme  are  the  “rapid  acquisition”  programs  that  meet  immediate  needs  and 
therefore  should  be  acquired  using  a  low-risk,  predictable  approach  with  well-defined,  currently 
available  technologies.  Those  programs  are  usually  not  MDAPs  though  there  have  been 
exceptions — notably  the  Stryker  and  the  Mine-Resistant  Ambush  Protected  (MRAP)  programs. 
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Most  MDAPs  fall  between  these  extremes,  and  for  most  of  them,  getting  it  right  is  more 
important  than  getting  it  soon.  Undue  emphasis  on  cycle  time  and  predictability  of  schedule  can 
inhibit  pursuit  of  programs  that  are  aimed  at  difficult  objectives  that  take  more  time  to  achieve. 
However,  that  does  not  mean  being  complacent  in  requiring  those  programs  that  aim  high  be  well- 
founded  on  clear  assessment  of  underlying  assumptions  regarding  technical  risk,  and  have  a  well- 
formulated  program  strategy  to  address  that  risk. 

In  those  instances  in  which  a  program  must  reach  for  levels  of  performance  that  stress  the 
state  of  art,  then: 

•  The  rationale  in  terms  of  future  threat  or  compelling  operational  need  should  be  clearly 
identified. 

•  Risks  should  be  explicitly  defined,  independently  verified,  and  made  clear  to  all 
management  levels. 

-  There  should  be  well-defined  approaches  pursued  to  reduce  the  risk  by 
carefully  maturing  the  technologies  and  addressing  their  integration  into  the 
system  as  the  system  evolves  over  time. 

•  The  means  for  accommodating  technology  improvement  and  change  into  the  system 
as  it  develops  and  is  fielded  overtime  should  be  a  clear  part  of  the  acquisition  strategy. 

•  The  program  should  receive  high-level  oversight  throughout. 
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I.  Introduction  and  Research  Focus 


Current  interest  in  acquisition  program  cycle  times  originated  with  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics  (USD(AT&L))  Better  Buying  Power  2.05 
initiative.  Better  Buying  Power  3.0, 6  recently  promulgated,  continues  an  emphasis  on  program  cycle 
times,  but  an  even  greater  concern  is  that  of  the  cycle  time  in  technological  innovation: 

Emphasize  technology  insertion  and  refresh  in  program  planning.  This  initiative  covers 
both  the  demand  side  (programs)  and  the  supply  side  (science  and  technology  projects). 
Because  of  the  pace  at  which  the  technology  associated  with  digital  processing,  radio 
frequency  devices,  optics,  and  networks  (among  others)  is  moving,  the  Department 
cannot  hope  to  keep  up  using  traditional  acquisition  approaches.  We  have  to  design  our 
acquisition  plans  to  account  for  periodic  technology  refresh  cycles  on  a  much  faster  time 
scale.  In  some  cases,  we  may  completely  replace  earlier  versions  of  end  products  (e.g., 
some  tactical  radios),  while  in  other  cases,  we  must  plan  and  design  for  periodic 
upgrades,  sometimes  while  development  is  still  in  progress  (e.g.,  F-35).  In  addition,  we 
need  to  ensure  that  our  early  research  and  development  (R&D)  investments  are  aligned 
as  much  as  possible  with  insertion  opportunities  in  the  products  we  are  likely  to  acquire. 

This  requires  a  tighter  connection  between  our  Science  and  Technology  communities  and 
our  development  programs. 

It  is  clear  from  this  extract  that  the  concern  is  more  than  just  the  amount  of  time  from  concept  to 
fielding  of  an  acquisition  program.  It  could  be  said  that  in  Better  Buying  Power  3.0  time  is  of  the 
essence — but  specifically  time  to  product  is  seen  as  highly  related  to  technological  capabilities. 

The  current  task,  shown  in  Figure  1,  is  focused  primarily  on  how  program  schedules  are 
established  and  executed,  though  our  investigations  have  led  to  some  insights  into  issues  of  broader 
concern  regarding  technology  capabilities  and  user  requirements. 


5  Frank  Kendall,  Memorandum  for  Defense  Acquisition  Workforce,  SUBJECT:  “Better  Buying  Power  2.0:  Continuing 
the  Pursuit  for  Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Washington,  DC:  Department  of  Defense 
(DOD),  November  13,2012. 

6  Frank  Kendall,  Memorandum  SUBJECT:  “Implementing  Directive  for  Better  Buying  Power  3.0 — Achieving 
Dominate  Capabilities  through  Technical  Excellence  and  Innovation,”  (Washington,  DC:  Department  of  Defense, 
April  09,  2015. 
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Specific  Tasks 

In  coordination  with  the  sponsor,  select  a  small  set  of  programs  for  in- 

depth  investigation  of: 

1.  Requirements  definition  and  schedules: 

-  To  what  extent  does  the  requirements  process  specify  lOCs  and 
FOCs  in  defining  program  requirements?  When  they  are  specified, 
what  factors  drive  setting  those  benchmarks? 

-  At  what  point  in  the  acquisition  process  do  schedule  parameters  get 
set?  What  role  does  the  requirements  community  play  in  those 
decisions? 

-  When  programs  miss  scheduled  milestones,  what  role  does  the 
requirements  community  play  in  adjusting  the  program? 

2.  Acquisition  schedule  development: 

-  How  are  schedules  for  acquisition  programs  set  and  how  are  they 
changed?  What  are  these  schedules  and  changes  based  on? 


Figure  1.  Subtasks 
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II.  Background 


The  term  “cycle  time”  in  reference  to  the  defense  acquisition  system  has  been  in  use  for  many 
years.7  However,  we  were  unable  to  find  an  official  definition  of  what  is  meant  by  “cycle  time”  in  the 
context  of  an  acquisition  program.  Better  Buying  Power  2.0  indicates  that  cycle  time  means  the  time 
“it  takes  to  bring  a  product  from  concept  to  fielding.”8  This  is  consistent  with  a  common 
understanding  that  the  “cycle  time”  refers  to  the  amount  of  time  it  takes  DOD  to  field  new  capabilities, 
particularly  MDAPs.  More  precisely,  our  research  is  taking  cycle  time  to  mean  the  time  between  the 
specification  of  a  requirement  for  a  military  capability  that  entails  a  major  system  development,  which 
normally  is  the  Materiel  Development  Decision  (MDD),  and  the  time  that  such  capabilities  are 
fielded.  As  we  will  elaborate  below,  even  this  leaves  room  for  some  ambiguity,  as  both  the  start  and 
end  points  can  be  elusive. 

In  particular  when  “cycle  time”  is  used  as  a  measure  with  specific  values,  interpreting  those 
values  depends  on  the  measure’s  definition.  The  2013  and  2014  DOD  reports  on  the  perfonnance  of 
the  defense  acquisition  system  do  not  state  a  definition,  but  it  is  clear  from  the  text  that  cycle  time  is 
interpreted  as  the  length  of  development  contracts.9  Even  though  that  is  clear,  the  report  does  not 
actually  state  what  development  contract  is  being  measured,  but  the  presumption  is  that  it  is  the 
primary  engineering  and  manufacturing  development  contract.  A  roughly  equivalent,  and  easier  to 
measure,  definition  is  the  time  between  Milestone  B,  the  decision  for  a  program  to  enter  Engineering 
and  Manufacturing  Development  (EMD),  and  achievement  of  an  Initial  Operational  Capability 
(IOC). 10  Based  on  that  measure,  contrary  to  commonly  held  perceptions,  acquisition  cycle  times  have 
on  average  not  been  growing  since  1980  (Figure  2). 11  However,  as  indicated  in  the  figure,  there  is 
wide  variation  in  results  over  programs.  It  is  arguable  that  this  is,  of  itself,  not  a  problem,  if  program 
schedules  are  determined  appropriately  to  meet  operational  requirements  and  if  those  schedules 
execute  as  planned.  However,  it  is  frequently  the  case  that  programs  fail  to  meet  their  established 


7  For  example,  the  1986  Packard  Commission  report  stated  that  .  .an  unreasonably  long  acquisition  cycle — ten  to 
fifteen  years  for  our  major  weapon  systems.  This  is  a  central  problem  from  which  most  other  acquisition  problems 
stem.”  A  Quest  for  Excellence ,  Final  Report  to  the  President  by  the  President’s  Blue  Ribbon  Commission  on  Defense 
management,  June  1986,  p.47. 

8 

See  http://bbp. dau.mil/bbp4focus.html. 

9  Performance  of  the  Defense  Acquisition  System,  2014  Annual  Report,  Washington,  D.C.:  Under  Secretary  of 
Defense,  Acquisition,  Technology  and  Logistics,  (USD(AT&L)),  June  13,  2014. 

10  Obviously,  both  these  measures  will  reflect  cycle  times  that  are  much  shorter  than  the  ones  suggested  in  the  previous 
paragraph.  See  footnote  12  on  page  6. 

1 1  Based  on  IDA  analysis  of  data  provided  by  OUSD(AT&L). 
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schedules.  Figure  3  (which  focuses  on  “bad  actors” — programs  executing  more  than  24  months 
behind  schedule)  shows  that  a  substantial  number  of  programs  have  had  long  delays  in  fielding. 12 
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Figure  2.  Time  between  Milestone  B  and  Initial  Operational  Capability  for  MDAPs 

since  1980 

Moreover,  schedule  growth  is  frequently  accompanied  by  cost  growth  and  in  many  cases  is 
indicative  of  technology-related  problems  in  development.  An  analysis  conducted  for  the  Office  of 
Director,  Operational  Test  and  Evaluation,  of  the  reasons  for  delay  in  1 15  historical  MDAPs  revealed 
that  delays  usually  have  multiple  causes;  however,  for  87  of  the  115  programs,  at  least  one  cause  of 
delay  was  a  performance  problem  discovered  during  testing.13  A  reasonable  (but  unconfirmed) 
hypothesis  is  that  many  such  performance  problems  had  root  causes  in  the  technological  solutions 
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12 

The  degree  to  which  those  delays  caused  operational  problems  has  not  been  determined  (and  is  not  within  scope  of 
the  present  study). 

This  analysis  can  be  found  on  the  Office  of  Director,  Operational  Test  and  Evaluation  website  at 
http://www.dote.osd.mil/pub/presentations/ProgramDelaysBriefmg2014  8 Aug  Final-77u.pdf 
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implemented  in  the  programs — that  is,  the  technological  approach  could  not  be  realized  within  the 
time  and  cost  estimates.14 


Figure  3.  Defense  Acquisition  Programs  from  2000,  with  Greater  Than 

24  Months  Slippage 


These  data  raise  issues  concerning  program  schedules  that  bear  further  assessment: 

•  Many  MDAPs  are  fielded  significantly  later  than  planned: 

-  What  are  the  processes  for  establishing  schedules  in  acquisition  programs? 


14  There  is  significant  support  for  the  hypothesis  that  ambitious  programs  that  require  development  of  crucial,  high-risk 
advanced  technologies  to  meet  stressing  performance  objectives  have  had  substantial  program  schedule  delays.  See 
for  example,  Riposo,  McKernan,  and  Duran,  “Prolonged  Cycle  Times  and  Schedule  Growth  in  Defense  Acquisition: 
A  literature  Review,”  The  RAND  Corporation.  2014,  p.  15.  Also,  several  PARCA  “Root  Cause”  studies  of  MDAPs 
that  had  critical  Nunn-McCurdy  breaches  cite  technology  readiness  issues  as  a  prime  cause  of  cost  growth,  including 
ATIRCM,  CMWS,  Apache  Block  III,  DDG-1000,  and  JTRS-GMR.  (See  Diehl,  Richard,  et  al,  Root  Causes  of  Nunn- 
McCurdy  Breaches— A  Survey  of  PARCA  Root  Causes  Analyses  2010-2011,  IDA  Paper  P-491 1,  Alexandria,  VA: 
Institute  for  Defense  Analyses,  August  2012.)  (Most  of  those  systems  also  experienced  significant  schedule  growth.) 
More  support  can  be  found  in  the  OT&E  report,  “Reasons  Behind  Program  Delays:  2014  Update,” 
(http://www.dote.osd.mil/pub/presentations/ProgramDelaysBriefmg2014_8Aug_Final-77u.pdf)  previously  cited. 


5 


—  To  what  extent  are  explicit  tradeoffs  between  perfonnance,  cost,  and  schedule  to  meet 
required  delivery  dates? 

-  What  are  the  consequences  of  delayed  deliveries  of  capabilities? 

•  MDAP  schedule  delays  are  characterized  by  wide  variances  and  outliers,  which  may 
suggest  major  problems  with  certain  types  of  systems, 

•  Acquisition  programs  that  stretch  the  technological  state  of  art  often  have  impacts  on 
schedule  and  cost. 

—  Presumably  less  ambitious  capabilities  could  be  delivered  sooner  and  with  less  risk. 

•  But,  would  those  reduced  capabilities  meet  operational  needs,  both  near-tenn 
and  longer-term?15 

—  Thus  the  question  of  whether  effective  tradeoffs  are  made  between  schedule  and 
capabilities  (and  if  so,  how  and  by  whom?) 

These  issues  became  the  basis  for  the  present  study. 

A.  Approach 

Figure  4  is  an  overview  of  the  “front-end”  of  the  process  for  initiating  an  MDAP.  The  MDD 
is  normally  the  fonnal  entry  point  into  the  DOD  acquisition  process  and  the  point  at  which  programs 
are  governed  by  the  primary  DOD  acquisition  process  issuances,  DODI  5000.02  and  their  DOD 
Component  equivalents.  That  time  is  arguably  the  appropriate  start  point  for  an  acquisition  program16 
with  IOC  being  the  end  point  (though  a  case  could  be  made  for  Full  Operational  Capability  (FOC)). 

While  an  MDAP’s  schedule  is  not  engraved  in  stone  until  an  Acquisition  Program  Baseline  is 
established  at  the  Milestone  B  entry  into  EMD,  it  is  essential  that  firm  ideas  on  schedule  be  set 
earlier — certainly  by  Milestone  A  entry  into  the  Technology  Maturation  and  Risk  Reduction  (TMRR) 
phase. 17  The  technology  selected  at  Milestone  A  must  be  sufficiently  mature  for  a  high  probability  of 
success  in  further  maturation  to  the  point  where  a  producible  engineering  design  can  be  developed  at 
the  end  of  the  TMRR  phase.  The  Initial  Capabilities  Document  (ICD)  must  reflect  a  reasonably 


15  If  the  answer  is  yes  for  near  term  but  no  for  longer  term,  then  an  evolutionary  acquisition  approach  is  indicated. 

16  There  are,  however,  several  technical  issues  with  this  definition.  One  is  that  some  programs  bypass  the  MDD 
decision  point  and  go  directly  to  a  Milestone  A,  Milestone  B,  or  even  Milestone  C.  A  practical  issue  is  that  the  MDD 
date  is  not  systematically  captured  in  an  accessible  database.  Although  the  instructions  for  SAR  preparation  state  that 
the  schedule  section  should  contain  all  major  milestone  decision  points,  none  of  the  SARs  checked  contained  any 
entries  earlier  than  Milestone  B. 

1 7 

See  Gene  Porter,  C.  Vance  Gordon,  R.  Royce  Kneece,  Improving  the  "Front-End”  of  the  DOD  Acquisition  Process, 
IDA  Paper  P-4710,  Alexandria,  VA:  Institute  for  Defense  Analyses,  June  2011,  for  an  extensive  discussion  on 
initiation  of  MDAPs. 
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accurate  understanding  of  what  technologies  can  be  sufficiently  matured  to  provide  the  needed 
capabilities.  Typically,  a  trade-off  between  the  level  of  advancement  of  technologies  and  the 
operational  needs  must  be  made  so  that  a  program  can  be  initiated  that  will  be  both  successful  and 
meet  operational  needs  to  the  extent  possible  (such  tradeoffs  are  indicated  by  the  dotted  arrows  on 
Figure  4).  An  approximate,  initial  program  schedule  should  be  an  output  of  that  process. 


How  are  Schedules  Set  for  MDAPs? 
The  Front-end  is  Crucial 


Develop  Requirements 
Services,  Joint  Staff 

Capabilities  Based  Analyses  (CBAs) 
•Threat 

•Combatant  Commander  priorities 
•Current/programmed  capabilities 
•  ->  Identify  capability  gaps 

(and  when  projected?) 


ICD 


Materiel 

Dev. 

Decision 


CDD 


Analysis  of  Alternatives  (AoA) 

•  Trade  offs 

•  Capability  vs.  cost 

•  Capability  vs.  schedule 
(How  are  they  made?) 


I 


Technology  Maturation/ 
Risk  Reduction 

•  Can  technology  required  for 
preferred  solution  be 
developed  and  produced  in 
time  to  meet  required  IOC ? 


Preferred  Alternative 


1 


Milestone  A 


Figure  4.  Process  for  Initiating  Major  Defense  Acquisition  Programs 

The  approach  taken  by  this  research  is  to: 

•  Determine  how  schedules  are  set  by  reviewing  JCIDS 1 8  and  acquisition  system  documentation 
on  the  schedule  setting  processes  and  by  conducting  interviews  with  personnel  in  both 
requirements  communities  (Joint  Staff  and  Services)  and  acquisition  communities  (program 
managers,  program  executive  officers,  headquarters  staffs) 


18  ....  . 

Joint  Capabilities  Integration  and  Development  System — The  Joint  Staffs  process  for  review,  approval  of  DOD 
Component  initiation  of  new  acquisition  programs,  and  oversight  of  acquisition  programs  once  launched. 
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•  Coordinate  with  the  sponsor  to  select  a  small  set  of  MDAPs  for  in-depth  investigation  of  the 
drivers  that  determined  the  schedule  benchmarks  specified  in  the  initial  requirements  and 
acquisition  documents 

•  For  the  programs  selected,  determine:  1)  the  inputs  used  to  set  times  by  which  stated 
requirements  are  needed,  2)  how  those  times  affected  the  setting  of  program  schedules,  and 
3)  within  the  program  management  process,  what  information,  analytical  tools  and  decision 
criteria  are  used  for  setting,  monitoring,  and  assessing  schedules  during  program  execution 

•  Identify  programs/acquisition  organizations  that  provide  examples  of  “best  practices”  for 
oversight  and  management  of  schedules 

B.  Selecting  Programs  for  Detailed  Investigation 

The  study  team  reviewed  the  “universe”  of  recent  and  active  MDAPs  to  identify  good  candidates 
for  more  detailed  examination.  The  initial  approach  was  to  select  a  mix  of  both  older,  more  mature 
programs  and  newer  programs  in  order  to  address  both  questions  about  the  ability  of  MDAPs  to 
execute  the  schedules  a  well  as  questions  about  how  schedules  are  established  to  begin  with.  This 
top-level  screening  is  shown  in  Figure  5. 

However,  in  an  interim  review  with  the  sponsor,  a  greater  focus  on  programs  that  began  under 
the  current  acquisition  team  was  requested,  to  establish  a  common  timeframe  with  interest  in  more 
current  versus  “historical”  programs.  With  that  in  mind,  the  following  programs  were  selected: 

1 .  Anny: 

•  Common  Infrared  Countermeasures  (CIRCM) 

•  Integrated  Force  Protection  Capability  (IFPC) 

2.  Navy 

•  Next  Generation  Jammer  (NGJ) 

•  Air  and  Missile  Defense  Radar  (AMDR) 

3.  Air  Force 

•  3-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

•  F-22  Increment  3.2B  Modernization 

The  more  current  programs  compared  to  the  initial  candidates  are  generally  smaller  scale  and 
focused  more  on  electronics  and  force  protection  systems  with  no  major  platfonns.  Since  all  of  these 
programs  are  several  years  from  IOC,  we  could  not  assess  whether  their  schedules  as  established 
could  actually  be  executed. 
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Time  Phase  of  System 

Service 

Platform/  System 
Type 

New 

(A>B) 

Current 

(>B,  but  <IOC) 

Fielded 

(IOC+) 

Low-Tech 

Complexity 

Medium-Tech 

Complexity 

High-Tech 

Complexity 

Low-Tech 

Complexity 

Medium-Tech 

Complexity 

High-Tech 

Complexity 

Low-Tech 

Complexity 

Medium-Tech 

Complexity 

High-Tech 

Complexity 

Army 

Rotary  Wing 

AH-64E  Apache 

UAV 

MQ-1C  Gray  Eagle 

Ground  Systems 

AMPV 

PIM 

Electronic 

CIRCM 

Munitions/Missiles 

IFPC 

Excalibur 

Navy 

Fixed  Wing-  Large 
Aircraft 

E-2D 

UAV 

UCLASS 

MQ-4C  Triton 

Ships 

ssc 

LCS 

Radar/Electronic 

NGJ 

AMDR 

Munitions/Missiles 

AGM-88E 

Air  Force 

Large  Aircraft 

KC-46A 

Combat  Aircraft 

F-22  Incr  3.2B 

UAV 

MQ-9  Reaper 

Satellites/ 

Space/C3l 

3DELRRS 

FAB-T 

WGS 

SBIRS  Follow-on 

Munitions/Missiles 

HTM 

GBSD 

Joint 

Combat  Aircraft 

F-35 

Ground  Systems 

JLTV 

C3I 

JALN 

Munitions/Missiles 

JAGM 

Abbrevaition  Key: 

AMPV:  Armored  Multipurpose  Vehicle 

PIM:  Paladin  Improvement  Program 

SSC:  Ship-to-Shore  Connector 

E-2D:  Advanced  Hawkeye  Aircraft 

UCLASS:  Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike 

HTM:  Hard  Target  Munition 

LCS:Littoral  Combat  Ship 

SBIRS:  Space-Based  Infrared  System 

FAB-T:  Family  of  Beyond-line  of  sight  Terminal  WGS:  Wideband  Global  Satellite 

GBSD:  Ground-Based  Strategic  Deterrent 

JLTV:  Joint  Light  Tactical  Vehicle 

JALN:Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 

JAGM:  Joint  Air-to-Ground  Missile 

C3I:  Command,  Control,  Communications  and  intelligence 

Figure  5.  Screening  of  Candidate  Programs  by  Service  and  Platform  Type 
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III.  Overall  Findings 


To  ground  this  assessment  from  the  standpoint  of  requirements  setting,  an  initial  discussion 
was  held  with  the  Office  of  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
OUSD(AT&L)  Director  of  Joint  Operations  Support,  attended  by  representatives  from  the  Joint  Staff, 
Force  Structure,  Resources,  and  Assessment  Directorate  (J-8).  This  was  followed  by  a  more  in-depth 
discussion  with  the  J-8  staff  regarding  the  overall  operations  of  the  Joint  Requirements  Oversight 
Council  (JROC)19  with  regard  to  schedule  setting  and  monitoring.  These  meetings  indicated  that 
initial  MDAP  schedules  are  determined  by  the  sponsoring  DOD  Component — usually  one  of  the 
Military  Services.  J-8  personnel  then  provided  the  IDA  team  points  of  contact  for  each  Service  for 
further  discussions. 

Subsequent  interviews  with  points  of  contact  for  Service  requirements  and  headquarters 
acquisition  staffs  revealed  there  have  been  some  fairly  recent  changes  in  the  way  that  new  acquisition 
programs  are  established.  With  recent  issues  regarding  long  delays  and  outright  cancellations  in 
several  high-visibility  programs,  there  is  a  strong  appreciation  in  the  Service  acquisition  communities 
of  the  need  to  change.  In  fact,  the  Navy  and  Air  Force  in  particular  have  made  efforts  to  tighten 
linkages  between  the  requirements,  acquisition,  and  resource  communities  early-on  in  the  program 
development  process.  In  addition,  constraints  on  cost  and  schedule  have  been  imposed  by  upper 
management  that  limit  the  trade  space  in  acquisition  program  development,  specifically: 

—  Better  Buying  Power  1 .0  made  affordability  caps  (a  cost  constraint)  equivalent  to  a 
Key  Perfonnance  parameter  (KPP)20 

-  The  Navy’s  Chief  of  Naval  Operations  has  addressed  the  importance  of  schedule, 
essentially  equating  it  to  a  KPP21 

19The  JROC  has  a  statutory  responsibility  to  identify,  validate,  and  prioritize  joint  warfighting  requirements.  The  JROC 
is  chaired  by  the  Vice  Chief  of  the  Joint  Staff  with  membership  by  each  of  the  Military  Service  vice  chiefs.  The 
JROC’s  role  in  the  acquisition  process  is  to  identify  and  assess  capability  needs,  priorities,  and  associated 
performance  criteria. 

20 

“  Ashton  B.  Carter,  “Memorandum  for  Acquisition  Professionals  Subject:  Better  Buying  Power:  Guidance  for 

Obtaining  Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Washington,  DC:  United  States  Department  of 
Defense,  September  14,  2010).  The  first  provision  is  to  “Mandate  affordability  as  a  requirement.” 

“  Chief  of  Naval  Operations,  “Mandatory  Navy  Key  Performance  Parameters  for  Cost,  Schedule,  and  Space,  Weight, 
Power,  and  Cooling  Margins,”  August  1,  2013,  The  memo  actually  says,  “The  goal  of  this  memorandum  is  to 
improve  the  Navy’s  focus  on  attributes  that  will  be  increasingly  important  in  the  current  and  emerging  fiscal  and 
security  environment.  First,  enforcement  of  cost  and  schedule  requirements  is  essential.  Placing  these  attributes  on 
par  with  traditional  performance  attributes  will  encourage  trades  between  cost,  schedule  and  performance  to  deliver 
programs  within  likely  fiscal  constraints  while  remaining  relevant  to  emerging  threats  and  opportunities.”  Thus,  the 
memo  encourages  the  kind  of  tradeoffs  necessary  to  achieve  schedules  when  it  is  critical  to  do  so  (but  not  when  it 
isn’t). 
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Both  of  these  management  edicts  seek  to  put  more  teeth  into  program  funding  and  scheduling 
parameters  with  the  intent  to  make  programs  adhere  more  closely  to  their  stated  cost  and  time 
estimates.  There  is,  however,  a  danger  that  this  greater  emphasis  on  developing  and  employing 
realistic  program  costs  and  schedules  will  discourage  the  development  and  use  of  advanced 
technologies  needed  to  meet  more  stressing  performance  requirements,  when  that  objective  might  be 
more  important  than  achieving  a  predetermined  (and  possibly  somewhat  arbitrary)  schedule  or  cost 
objectives  that  were  perhaps  unrealistically  set.  That,  of  course,  is  a  complicated  and  open-ended 
issue  about  which  much  has  been  written. 

The  intent  of  the  Service  up-front  efforts  to  better  link  their  requirements,  acquisition  and 
resource  organizations  is  to  achieve  realism  across  these  three  domains.  However,  given  that  intent, 
there  is  little  evidence  from  the  interviews  conducted  and  programs  examined  that  trades  were 
considered  that  lowered  requirements  in  favor  of  accelerating  schedules.  It  is  important  to  note  that 
these  initiatives  to  discipline  the  program  development  process  are  fairly  recent,  and  the  fruits  of  these 
initiatives  may  not  be  evident  for  some  time.  Currently,  under  highly  constrained  resources  and 
attendant  higher-level  concerns  from  both  Service  and  Office  of  Secretary  Defense  (OSD)  executives, 
there  is  strong  motivation  for  such  “realism.” 

United  States  Code  (Title  10  USC  2366a)  requires  that  program  managers  provide  notification 
if  their  program  is  likely  to  exceed  its  prescribed  schedule  by  more  than  25%.  Specifically,  pursuant 
to  10  USC  181  (which  specifies  the  duties  of  the  JROC)  once  the  Milestone  Decision  Authority 
(MDA)  certifies  various  criteria  at  Milestone  A,  the  program  manager  must  notify  the  MDA  if,  at  any 
time  prior  to  Milestone  B,  it  is  determined  that  there  will  be  insufficient  time  to  deliver  an  IOC  and 
that  the  program  is  likely  to  exceed  the  schedule  objective  by  more  than  25  percent. 

This  raises  the  basic  question:  How  could  we  tell  if  a  program  will  exceed  the  IOC  objective 
established  by  the  JROC?  It  turns  out  that  tracking  the  status  of  programs  is  fraught  with  ambiguity. 
First,  routine  program  reporting  requirements  prior  to  Milestone  B  are  scant,  and  may  be  inadequate 
for  tracking  status  relative  to  these  notification  parameters.22  Secondly,  an  analysis  conducted  by 
OUSD(AT&L)  determined  that  roughly  two-thirds  of  the  time,  an  IOC  target  was  never  established 
in  the  ICDs  for  the  programs  reviewed.  (That  is  consistent  with  the  findings  of  this  study  for  the  small 
sample  of  programs  investigated.)  Generally,  schedule  dates  are  not  set  firmly  until  approval  of  the 
Capabilities  Development  Document  (CDD)  which,  according  to  CJSC  Manual  3 170, 23  must 


22 

If  reporting  is  required  by  pre-Milestone  B  contracts,  the  Earned  Value  Management  Reports  should  provide  some 
early  warning  on  schedule  delays. 

23 

“  Joint  Staff,  “Manual  for  the  Operation  of  the  Joint  Capabilities  Integration  and  Development  System,”  February.  12, 
2015. 
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"Specify  the  target  date  for  IOC  and  FOC  attainment..."  (The  ICD  is  also  required  to  address  IOC, 
but  with  less  specificity.)  So  at  the  time  of  the  Analysis  of  Alternatives  (AoAs)  and  Milestone  A 
starting  technology  maturation,  for  the  large  majority  of  programs  a  requirements-driven  schedule  is 
not  specified.  See  Appendix  B  for  a  summary  of  extracts  from  Chainnan,  Joint  Chiefs  of  Staff,  and 
DOD  issuances  regarding  setting  schedules  and  making  tradeoffs  involving  schedules. 

A.  Questions  for  Program  Offices 

To  focus  on  specific  concerns  regarding  setting  of  schedules  for  individual  MDAPs,  the  IDA 
study  team  provided  a  detailed  set  of  questions  to  each  of  the  program  offices  and  responsible  offices24 
for  the  selected  programs.  These  questions  guided  discussions  and  several  programs  in  fact  provided 
written  responses.  The  questions  submitted  are  listed  below: 

Setting  Acquisition  Program  Schedule  Dates: 

1 .  At  what  time  were  acquisition  schedule  dates  for  your  program  first  specified?  What 
were  the  key  dates? 

2.  In  what  official  document  were  the  dates  specified  and  under  what  approval  authority 
was  it  promulgated? 

3.  What  requirements  documents  supported  specification  of  those  dates? 

4.  What  role  did  the  requirements  community  play  in  establishing  the  schedule  dates  for  the 
acquisition  program? 

5.  To  what  extent  were  tradeoffs  between  schedule  and  perfonnance  and/or  costs  made  in 
establishing  those  dates  and  how  were  they  done?  What  tools  were  used  in  analysis 
supporting  such  tradeoffs? 

6.  How  do  you  detennine  the  feasibility  of  development  schedules?  What  role  do 
contractors  play  in  making  such  estimates?  How  are  schedule  parameters  reflected  in 
Requests  for  Proposals,  proposals,  and  contracts? 

Revising  Acquisition  Program  Schedule  Dates: 

1 .  Provide  a  chronology  of  any  changes  to  key  schedule  dates  since  originally  approved 
and  briefly  discuss  the  reasons  for  the  changes. 

2.  What  role  has  the  requirements  community  played  in  revisions  to  schedule  dates? 

3.  To  what  extent  were  tradeoffs  between  schedule  and  perfonnance  and/or  costs  made  in 
the  change  process? 


24  E.g.,  Staff  action  officers  for  programs. 
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Funding-related  Issues: 

1 .  Has  the  program  always  been  fully  funded  by  the  Component? 

2.  What  were  the  impacts  of  resource  management  decisions,  funds-execution-related  cuts, 
Congressional  cuts,  Continuing  Resolutions,  etc.? 

B.  Findings  from  Review  of  Selected  MDAPs 

1.  Overall  Insights 

Insights  emerged  from  the  programs  investigated. 

•  Most  of  the  programs  examined  are  in  response  to  requirements  first  stated  in  the  2004-2010 
timeframe.  Most  IOCs  are  in  the  2020+  timeframe 

•  Most  program  requirements  are  near-tenn  and  threat-driven,  though  obsolescence  also  has 
been  a  factor. 

•  Several  programs  appear  to  have  a  requirements-capabilities  mismatch;  the  problems  appear 
related  to  unavailability  of  technology  needed  to  achieve  the  necessary  perfonnance  relative 
to  the  defined  threat.  Capabilities  will  be  available  10 years  or  more  after  the  threat  emerged. 

These  observations  raise  questions  concerning  how  threats  are  defined  and  when  they  will 
emerge  operationally. 

2.  AMDR  and  NGJ 

Figure  6  displays  insights  for  the  AMDR  and  NJG  programs.  For  the  aspects  under 
consideration  in  this  task,  these  two  Navy  programs  are  structurally  very  similar,  thus  their  findings 
have  been  combined.  Both  were  driven  by  the  need  to  employ  Gallium  Nitride  (GaN)  chip  technology 
to  achieve  the  required  performance  within  space,  weight,  and  power  (SWAP)  constraints. 
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NGJ  and  AMDR 

•  Requirements  are  strongly  "near-term"  threat-driven 

-  New  capabilities  of  potential  adversaries: 

•  Surface-to-air  missiles— longer  range,  more  sophisticated 

•  Radars— range,  frequency  bands,  counter-antiradiation  missile  capabilities 

•  Tactical  ballistic  missiles  and  supersonic  cruise  missiles 

•  Requirements  validated  in  studies  in  the  2006-2010  timeframe 

•  Both  systems  need  greater  power  and  sensitivity  and  have  space,  weight, 
and  power  constraints 

•  The  technological  solution  is  Gallium  Nitride  (GaN)  integrated  electronics 
(e.g.,  transmit/receive  modules) 

-  GaN,  at  the  time  program  formulated,  was  an  emerging  technology  in  DoD 
research  and  development  activities 

-  Issue  of  technical  risk  versus  alternatives  that  could  meet  needs  were  matters 
of  judgment 

-  DoD  programs  to  accelerate  GaN  paralleled  programs  that  would  employ  it 

These  programs  illustrate  the  need  to  address  technology  maturity  in 
program  development  and  planning 

•  Must  be  addressed  prior  to  Milestone  A 

•  Are  appropriate  processes  in  place  (early  systems  engineering  linked  to 
requirements  process)? 

•  How  were  /  are  technology  maturation  processes  considered  in  setting 
schedules  and  attendant  risks?  How  should  they  be? 


Figure  6.  Findings  for  AMDR  and  NGJ 

The  AMDR  program  replaces  two  radars  in  the  Aegis  system  on  future  production  of  DDG- 
5 1  destroyers.  The  new  radars  will  be  installed  on  the  Flight  III  DDG-5 1  s  beginning  with  the  FY2016 
procurement.  The  “in-yard  need  date”  was  originally  established  as  4th  quarter  FY2019  but  in  2013 
was  slipped  to  1st  quarter  FY2020.  That  requirement  is  the  primary  schedule  driver  for  the  program, 
since  any  slip  in  deliveries  would  result  in  what  could  be  an  expensive  slip  in  ship  production 
schedule.  The  program  has  three  major  components — an  S-band  radar,  an  X-band  radar,  and  a  radar 
control  system  and  interface  with  Aegis.  Importantly,  the  S-band  radar,  which  is  crucial  to  the  meeting 
the  program  requirements,  drew  upon  what  was  at  the  time  an  emerging  technology — GaN  integrated 
circuits.  These  are  needed  because  of  the  high  sensitivity  and  power  required  to  detect  incoming 
threats  at  sufficient  range  for  effective  intercept.  A  single  S-band  radar  employs  several  thousand 
GaN  transmit-receive  modules.  When  the  program  began,  GaN  technology  was  not  yet  available  and 
had  to  be  matured  to  meet  this  need.  Combined  efforts  of  the  DOD  through  Defense  Advanced 
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Research  Projects  Agency  and  Military  Service  technology  development  programs  with  selected 
electronics  contractors  succeeded  in  maturing  GaN  for  this  use. 

The  NGJ  program  replaces  some  of  the  capabilities  of  the  AN/ALQ-99  jammers  used  on  the 
EA-18G  “Growler”  electronic  warfare  aircraft.  Program  officials  stated  that  it  would  have  been 
desirable  to  have  the  NGJ  capabilities  when  the  EA-18Gs  were  first  fielded  in  late  2009.  However, 
the  NGJ  is  now  scheduled  for  first  delivery  in  -2020.  While  a  lack  of  sufficiently  high  priority  on  the 
part  of  the  Navy  was  cited  initially  as  the  reason  for  delay  by  Navy  personnel,  further  discussion  and 
investigation  indicates  that  the  availability  of  GaN  integrated  circuits  was  also  a  significant  driver. 
The  GaN  technology  enables  achieving  the  required  power  within  the  SWAP  constraints  of  the  host 
aircraft.  It  is  not  clear  that  this  technology  could  have  been  available  to  meet  the  2009  objective.  Use 
of  older  Gallium  Arsenide  technology  might  have  been  acceptable,  but  would  not  have  provided  as 
robust  a  capability. 

Both  of  these  programs  appear  to  have  followed  processes  that  qualify  as  “best  practices.” 

3.  3DELRR 

Figure  7  displays  insights  for  the  3DELRR  program.  This  program  has  a  long  and  complicated 
history.  The  initial  development  was  begun  by  the  U.S.  Marine  Corps  in  the  2003  timeframe.  The 
Marine  Corps  perceived  a  need  to  replace  their  AN/TPQ-59  airspace  control  radars.  The  Marine  Corps 
program,  known  at  the  time  as  the  Highly  Expeditionary  Long  Range  Air  Surveillance  Radar,  had  a 
projected  IOC  in  2012.  The  Air  Force  participated  in  the  program  in  a  limited  way  with  a  plan  to  buy 
the  radars  when  available  for  procurement. 

In  its  2008  Program  Objective  Memorandum,  the  Marine  Corps  decided  not  to  replace  the 
AN/TPQ-59  radars  and  proposed  to  terminate  the  program.  The  Air  Force  still  had  a  requirement  to 
replace  the  AN/TPQ-75  radars,  which  were  fielded  in  the  early  1970s  and  were  deemed  obsolete. 
Therefore,  the  Air  Force  took  over  the  lead,  while  the  Marine  Corps  continued  participation  with  a 
modified  objective  of  a  product  improvement  of  the  AN/TPQ-59.  The  Air  Force  acquisition  executive 
in  December  2007  signed  a  memorandum  to  establish  its  own  program.  In  February  2009,  the  Defense 
Acquisition  executive  designated  the  program  as  a  “special  interest”  MDAP  and  directed  a  joint  Air 
Force/Marine  Corps  program.  However,  in  August  2009,  despite  pressure  from  OSD,  and  in  light  of 
“fiscal  realities,”  the  Marine  Corps  withdrew  all  financial  participation  in  the  program,  stating  its 
intent  to  procure  the  Air  Force-developed  3DELRR  in  the  2025  timeframe.  A  resource  management 
decision  in  January  2011  provided  $24.1  million  in  FY2013  to  compensate  the  Air  Force  for  the 
Marine  Corps  withdrawal  (but  no  funding  in  the  out-years).  The  upshot  of  this  convoluted  process  is 
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that  3DELRR  was  delayed  by  six  years  with  the  original  IOC  of  2014,  set  in  2006,  now  being  2020. 25 
The  operational  impact  has  not  been  assessed. 


3DELRR 

•  Replaces  "obsolete"  AN/TPS-75  radar  used  for  surveillance  and  air  space 
control 

-  AN/TPS-75  (early  1970s)  still  operationally  effective.  Some  reliability  issues,  which 
may  be  overstated  due  to  issues  with  inaccuracies  in  the  readiness  reporting  system 

•  IOC  slipped  from  FY2014,  as  specified  in  2006,  to  FY2020  currently 

•  Drivers: 

-  Issues  with  joint  program  with  Marine  Corps  causing  change  in  lead  Service  (2007) 

-  Eventual  Marine  Corps  withdrawal  (2009-2010),  despite  OSD  pressures 

-  DCAPE  directs  continued  competition  in  TMRR  Phase  2;  competitor  added 

-  OSD  funding  reductions  in  February  2012 

-  OSD  non-concurrence  on  acquisition  strategy  and  source  selection  strategy  (2013) 

-  Award  protest  with  Air  Force  decision  to  re-evaluate  the  proposals  and  subsequent 
lawsuit  filed  by  Raytheon  (ongoing) 

•  Also  employs  GaN  technology  [not  clear  how  this  has  impacted  program] 

Total  delay:  6  years  (72  months);  largely  explained  by  above 
....but  one  can  ask  so  what?  Current  system  still  viable,  though 
expensive  to  support ...  But  not  effective  against  5th-generation 
aircraft,  small  UAVs,  or  ballistic  missiles 


Figure  7.  Findings  for  the  3DELRR  Program 

4.  Counter-IR  Countermeasure  (CIRCM)  System 

Figure  8  displays  insights  for  the  CIRCM  program.  This  program  is  developing  a  lightweight 
laser  countermeasure  system  for  smaller  helicopters.  The  device  will  use  a  laser  beam  to  confuse 
infrared  (IR)-guided  missile  seekers.  The  existing  protection  for  such  helicopters  is  the  Common 
Missile  Warning  System  (CMWS),  combined  with  the  AN/ALQ-144  flare  dispenser.  As  adversary 


25  •  •  • 

“  As  of  this  writing,  the  program  is  being  delayed  further  by  a  lawsuit  emanating  from  a  protest  of  the  award  of  the 
EMD  contract,  with  requested  funding  in  the  FY2016  President’s  budget  greatly  reduced.  Thus,  a  2020  IOC  is 
probably  no  longer  achievable. 
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IR-guided  missile  seekers  become  more  advanced,  they  are  able,  using  various  techniques,  to  ignore 
the  flares  and  continue  to  target  the  aircraft. 


CIRCM 

•  Army  program  with  Navy  participation  to  develop  a  lightweight  IR  missile 
countermeasure  system-improved,  lighter-weight  version  of  the  ATIRCM* 
system 

•  Laser-based  system  to  defeat  in-flight  missiles;  works  with  the  Common 
Missile  Warning  System,  which  detects  IR  missile  threats 

•  Existing  AN/ALQ-144  series  systems  (1980s)  not  capable  against  modern 
man-portable  IR  missiles— operational  losses  prompted  increase  in  priority 

•  Materiel  Development  Decision  in  2009 

-  Army  planned  to  go  directly  to  Milestone  B  [accelerated  acquisition] 

-  However,  using  a  Broad  Agency  Announcement,  Army  solicited  off-the  shelf  solutions 
and  received  equipment  from  five  vendors;  testing  showed  none  was  ready  for  EMD 

-  OSD  concluded  threat  dictated  a  need  date  of  FY2017  and  directed  trade-off  analyses  to 
meet  it;  Army  Program  Manager  asserted  that  meeting  that  date  not  possible 

-  Army  directed  to  prepare  for  an  Milestone  A 

•  Milestone  A  in  2011  approved  a  competitive  TMRR  phase 

•  In  July  2014,  an  Acquisition  Decision  Memorandum  approved  release  of 
EMD  Request  for  Proposals 

*  Advanced  Threat  Infrared  Countermeasures  used  on  CH-47s 


Figure  8.  Findings  for  the  CIRCM  Program 

The  Army  began  development  of  a  system  known  as  ATIRCM  (Advanced  Threat  IR 
Countermeasure)  in  1995.  That  program  has  a  long,  turgid  history.26  Originally  intended  to  protect 
all  Army  helicopters,  in  2003  it  was  scaled  back  to  only  procuring  systems  for  the  CH-47  fleet  because 
of  excessive  weight  (325  lbs.)  with  procurement  of  83  systems  completed  in  2009.  The  CIRCM  has 
a  weight-limit  threshold  KPP  of  120  lbs.  with  an  objective  of  65  lbs.  and  will  equip  the  Army’s  AH- 
64,  UH-60,  OH-58,  and  Marine  Corps  AH-1Z  fleets.  It  is  also  planned  to  replace  the  existing 
ATIRCMs  on  CH4-47s.  The  CMWS,  which  was  developed  in  conjunction  with  (and  also  works  with) 
the  ATIRCM,  detects  threat  missile  launches  and  provides  pointing  infonnation  to  the  laser  projector. 

In  2009,  the  CIRCM  was  initially  conceived  as  a  fast-track  acquisition — going  directly  into 
Milestone  B  using  what  were  to  be  off-the-shelf  solutions.  However,  when  the  Army  tested  the 
technologies  offered  by  potential  vendors  in  response  to  a  Broad  Agency  Announcement,  it 


26 

See  Davis,  Gregory,  et  al.  Root  Cause  Analysis  for  the  AT1RCM/CMWS  Program,  IDA  Paper  P-4601,  Alexandria, 
VA:  Institute  for  Defense  Analyses,  June  2010. 
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determined  that  none  was  ready  for  moving  into  EMD.  Therefore,  the  Army  revamped  the  program 
as  a  traditional  acquisition  with  a  Milestone  A  in  201 1,  approving  initiation  of  TMRR.  A  Milestone 
B  in  July  2014  approved  release  of  the  RFP  for  EMD.  Thus,  the  Anny  determined  that  instead  of 
moving  quickly  into  EMD  in  2009,  the  lack  of  sufficiently  capable  technology  delayed  EMD  for  live 
years. 

It  is  important  to  note  that  this  is  clearly  a  threat-driven  program  where  existing  helicopters 
are  vulnerable  and  have  been  lost  to  enemy  IR  missiles  in  Iraq  and  Afghanistan.  This  drove  the 
urgency  to  achieve  rapid  acquisition — but  technology  providing  needed  performance  within  the 
required  weight  limit  had  not  been  developed.  Under  these  circumstances  the  restructuring  of  the 
program  was  obviously  necessary,  but  the  question  to  be  asked  is  whether  an  earlier  technology 
maturation  program,  begun  for  example  in  2003  when  the  weight  issue  with  ATIRCM  emerged, 
would  have  provided  the  needed  technology  earlier. 

This  program  shows  that  the  objective  of  more  rapid  acquisition,  even  when  there  are  clear 
operational  consequences  of  not  doing  so,  must  be  done  with  clear  understanding  of  the  state  of  the 
underlying  technologies.  If  acquisition  is  to  be  accelerated,  then  careful  attention  must  be  paid 
investments  in  technology  maturation. 

5,  Integrated  Force  Protection  Capability  (IFPC)  Increment  2-1  Block  1 

Figure  9  displays  insights  for  the  IFPC  program.  This  system  currently  has  an  objective  of 
providing  fixed-point  and  “semi-fixed-poinf  ’  force  protection,  initially  against  cruise  missiles  (CMs) 
and  Unmanned  Air  Vehicles  (UAVs).  A  later  increment  (Block  2)  is  planned  for  the  same  launchers 
to  provide  protection  against  rockets,  artillery  and  mortars  (RAM).  (Block  3,  which  would  provide 
area  defense,  is  not  contemplated  to  start  until  post-2025.)  Initially  the  program  was  focused  on 
providing  a  more  robust  capability  to  RAM  as  an  existing  threat — more  robust  than  the  C-RAM 
system27  that  was  deployed  to  Iraq  to  protect  forward  operating  bases.  However,  it  was  determined 
that  available  technology  would  not  meet  that  requirement.  Therefore,  in  September  2012  the  Army 
Vice  Chief  of  Staff  refocused  the  program  on  protection  against  CMs  and  UAVs.  Indeed  the  Block  2 
aimed  at  RAM  is  not  scheduled  to  start  until  2019.  (A  lower-level  technology  program  is  continuing, 
and  the  Army  hopes  to  have  a  Milestone  A  for  Block  2  in  FY2019.) 


27 


Counter-RAM  system,  based  on  the  Navy  Phalanx  radar-directed  gun  system  for  close-in  protection  of  ships. 
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IFPC-Increment  2-Intercept  (Inc.  2-1) 

•  Began  as  a  requirement  to  counter  rockets,  artillery  and  mortars  (RAM) 

-  Various  ICDs,  Operational  Requirements  Documents,  and  CDDs  in  2002-2010  —  This  is 
an  existing  threat 

-  No  technical  solutions  appeared  to  meet  requirements 

•  In  January  2011  OSD  directed  Army  to  address  capability  gap  for  point  defense 
against  UAVs  and  cruise  missiles  (driven  by  Major  Contingency  Operations  threats) 

•  MDD:  Aug.  2011 

•  AoA  completed  April,  2013 

•  Army  defined  the  program  as  follows: 

-  Inc.  2-1  Block  I— counter  UAVs  and  cruise  missiles  (IOC  4QFY19) 

-  Incr.  2-1  Block  2  ( beginning  FY19)  is  to  counter  RAMs 

—  Incr.  2-1  Block  3  (FY27)  is  to  provide  area  defense  against  UAVs  and  cruise  missies 

•  At  Milestone  A,  Army  proposed  an  in-house  development  of  "Multi-Mission 
Launcher"  prototypes: 

-  Approach  is  to  "leverage  existing  sensors  and  mission  command... [and]  an  existing  interceptor  as  a 
design  reference  and  point  of  departure."  (Acquisition  Strategy,  06/2014) 

—  MS-A  approval  -  March  2014 

Appears  that  the  incremental  acquisition  strategy  won't  provide  response  to  key 
existing  threats  for  very  long  time.  How  will  that  impact  operations? 


Figure  9.  Findings  for  the  Integrated  Fire  Protection  Capability  Program 

IFPC  raises  interesting  questions  concerning  scheduling:  apparently  the  original  and  existing 
threat  driving  the  program  turned  out  to  be  too  difficult  to  meet  so  that  the  program  was  redirected  to 
what  appears  to  be  a  more  easily  attainable  technical  objective — but  not  the  originally  defined  threat. 
The  decision  also  appears  to  have  been  in  response  to  the  2012  policy  shift  away  from  irregular 
warfare  and  stability  operations  toward  major  contingency  operations. 

6.  F-22  3.2B  Modernization 

Figure  10  displays  insights  for  the  F-22  3.2  Modernization  Program.  This  program,  one  of  a 
series  of  modifications  to  modernize  the  Air  Force  F-22  Raptor  fighter/bomber  aircraft  fleet,  was 
designated  a  “special  interest”  MDAP  by  the  USD(AT&L)  in  December  2011.  The  program’s 
RDT&E  funding  exceeds  the  MDAP  threshold  ($500  million)  but  the  procurement  funding  does  not. 
Many  acquisition  programs  exceed  the  thresholds  specified  by  statute,  but  the  DOD  Components  have 
some  discretion  in  deciding  whether  to  designate  a  program  as  an  MDAP.  However,  the  Defense 
Acquisition  Executive  has  the  authority  to  designate  a  program  as  an  MDAP,  and  that  was  the  case 
here.  The  reasons  for  that  decision  have  not  been  documented. 
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F-22  3.2B  Modernization 

•  Part  of  a  long-term  program  to  maintain  and  upgrade  F-22 
capabilities— only  increment  designated  an  MDAP* 

•  Requirement  based  on  a  2007  Capability  Production  Document 
for  all  increment  3  modifications 

-  MDD  in  2011 

-  Acquisition  Decision  Memorandum  designated  MDAP  [rationale  not 
known] 

-  No  Milestone  A 

-  Milestone  B  June  2013 

•  Includes: 

-  Modifications  to  employ  AIM-120D  and  AIM-9X  block  2  missiles 

•  AIM-9X  block  2  lOCs  this  year;  rate  production  for  AIM-120D  this  year  (no  IOC  specified) 

-  New  geolocation  capability  (locates  emitters) 

-  Improved  electronic  protection 

-  Procurement  of  mod  kits  begins  in  FY16 

-  "Required  Assets  Available  (RAA)"  Sepember  2019  (means  first  unit 
equipped) 

*RDT&E  exceeds  threshold 


Figure  10.  Findings  for  the  F-22  3.2B  Modernization  Program 

The  primary  drivers  of  schedule  appear  to  be  the  modifications  to  enable  the  aircraft  to  carry 
the  AIM-120D  AMRAAM  and  AIM-9X  Sidewinder  Block  II  missiles.  These  missiles  will  be 
available  for  employment  this  year,  but  the  aircraft  modifications  will  not  be  started  until  2019,  so  the 
nation’s  pre-eminent  fighter  aircraft  will  not  be  capable  of  employing  the  most  advanced  air-to-air 
missiles  for  four  years  or  more.  Other  mods  include  the  ability  to  passively  locate  emitters  via 
coordinated  signal  intercept  by  two  or  more  F-22s.  This  technology  is  the  most  advanced  being 
implemented  in  this  modernization  program.  The  other  area  of  improvement  is  in  the  electronic 
protection  suite. 
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C.  Summary  and  Conclusions 

Most  of  the  programs  examined  are  in  response  to  requirements  first  stated  in  the  2004-2010 
timeframe  and  most  have  Initial  Operational  Capability  (IOC)  dates  in  the  2020-plus  timeframe.  Most 
requirements  are  near-term  threat-driven,  though  obsolescence  is  also  a  factor.  It  is  our  observation 
that  the  needed  capabilities  will  be  available  some  10  years  or  more  after  threat  emerged.  That  is,  the 
threats  these  programs  would  address  either  exist  now  or  will  emerge  into  operational  reality  before 
the  systems  are  fielded.  That  raises  questions  of  how  threats  are  defined  and  when  they  are  seen  as 
realized.  From  that  we  conclude  that  for  some  programs  there  appears  to  be  a  requirements- 
capabilities  mismatch;  where  the  problem  appears  to  be  the  unavailability  of  technology  needed 
relative  to  the  defined  threat. 

The  study  arrived  at  the  following  more  specific  findings: 

•  Requirements  inputs  on  IOC/schedules  are  difficult  to  pin  down — documentation  is  lacking 
or  difficult  to  obtain.  The  IOCs  reflected  in  acquisition  documents  were  drawn  from 
requirements  documents  drafted  between  Milestones  A  and  B28 — relatively  late  in  process- 
and  it  is  not  clear  whether  specified  IOCs  were  based  on  need  or  what  is  achievable.  JCIDS 
requirements  documents  are  archived  in  the  Knowledge  Management  and  Decision  System 
maintained  on  the  SIPRNET  by  the  Joint  Staff  (J-8).  That  system  is  difficult  to  use  and  does 
not  appear  to  contain  all  relevant  documents. 

•  The  team  was  not  successful  in  obtaining  several  Capabilities  Based  Assessments  (CBAs)  and 
ICDs  that  were  specific  for  systems  reviewed.  Only  one  system  (AMDR)  had  a  clear  statement 
(in  the  AoA)  of  when  specific  threat  capabilities  were  projected  to  be  operational. 

•  Program  schedule  setting  varies  in  rigor. 

—  The  Navy  appears  to  have  a  well-defined  process  (run  by  N829)  to  initiate 
programs  based  on  agreed  understanding  between  “requirers,”  developers,  and 
resource  planners. 

—  The  Air  Force  has  a  similar  process  but  with  a  lesser  role  for  resource  planners. 

—  For  programs  examined,  there  was  little  evidence  of  consideration  of  tradeoffs  of 
performance  requirements  and/or  costs  in  favor  of  accelerated  schedules. 

•  Focus  on  technology  maturation  as  part  of  early  acquisition  strategy  can  be  crucial. 


28 

“  Up  to  the  interim  version  of  DODI  5000.02  published  in  November  2013 — i.e.,  the  one  in  effect  when  all  the 

programs  investigated  were  initiated.  The  current  5000.02  requires  a  draft  Capabilities  Development  Document  prior 
to  Milestone  A. 

29 

“  Office  of  Deputy  Chief  of  Naval  Operations,  Resources,  Requirements  and  Assessments 
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—  The  AMDR  and  NGJ  programs  showed  that  technology-driven  programs  can  be 
successful  when  attention  is  focused  on  earlier  technology  maturation.  CIRCM  is  an 
example  of  the  apparent  failure  to  do  that. 

•  Tradeoffs  of  performance,  cost,  and  schedule.  If  it  is  deemed  essential  to  meet  firm  schedule 
dates  to  support  operational  or  other  critical  needs,  it  may  be  necessary  to  make  tradeoffs  of 
perfonnance  (especially  in  regard  to  the  technologies  employed)  and/or  costs  in  order  to 
achieve  the  schedule  needed.  Although  the  current  DODI  5000.02  contains  several  provisions 
requiring  tradeoff  analyses  among  cost,  schedule,  and  perfonnance,  there  is  no  explicit 
provision  to  consider  the  relative  urgency  of  operational  needs  in  making  such  tradeoffs.  An 
explicit  statement  along  those  lines  would  be  preferred.  Changes  also  may  be  needed  to  Joint 
Staff  procedures  in  order  to  identify  such  critical  need  dates. 

—  Appendix  B  documents  the  provisions  found  in  the  current  DODI  5000.02  that  address 
tradeoffs.  The  main  areas  most  relevant  hereto  are  in  1)  systems  engineering  and  2) 
performing  AoAs.  Under  the  “Development  Planning”  heading  of  Systems 
Engineering,  the  DODI  5000.02  contains  the  following  language: 

During  the  acquisition  life  cycle,  the  Program  Manager  will  conduct  systems  engineering  trade¬ 
off  analyses  to  assess  system  affordability  and  technical  feasibility  to  support  requirements, 
investment,  and  acquisition  decisions.  Systems  engineering  trade-off  analyses  will  depict  the 
relationships  between  system  life -cycle  cost  and  the  system’s  perfomiance  requirements,  design 
parameters,  and  delivery  schedules.  The  analysis  results  should  be  reassessed  over  the  life  cycle 
as  system  requirements,  design,  manufacturing,  test,  and  logistics  activities  evolve  and  mature. 
(Enclosure  3,  Paragraph  4a,  p.  82) 

The  second  sentence  of  this  provision  is  the  gist;  however,  it  would  be  desirable  to  add 
language  that  the  analysis  should  consider  performance  and  schedule  trades  to  best  meet 
user  needs  for  both  capabilities  and  timeliness,  in  light  of  the  readiness  of  the  critical 
technologies.30 

In  interviews  relating  to  the  programs  that  were  examined  in  depth,  evidence  of  such 
tradeoff  studies  was  not  identified  despite  specific  requests.31 

•  Funding  impacts:  It  is  difficult  to  discern  causes  from  effects  for  funding  reduction  impacts 
on  schedules. 

—  The  only  clear  case  identified  in  which  funding  issues  caused  program  delay  was 
NGJ,  which  incurred  a  double-hit  for  execution  delays  caused  by  award  protest. 


30 

As  noted  in  Appendix  B,  these  directive  provisions  pre-date  the  current  DODI  5000.02  in  the  form  of  a  “Directive 
Type  Memorandum”  dated  September  2010.  In  addition,  “corporate  memory”  was  found  to  be  incomplete — it  is 
quite  possible  that  such  trades  were  made  in  the  early  stages  of  programs  but  these  records  are  not  readily  available. 

31 

Though  not  identified  by  Air  Force  personnel,  a  reference  to  such  tradeoff  analysis  was  discovered  for  the  3DELRR 
program  in  the  Technology  Development  Strategy  May  2009  (page  9).  Air  Force  headquarters  was  unable  to  provide 
any  additional  information. 
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-  Several  programs  appeared  to  lack  priority  in  the  program/budgeting  process — 
3DELRR,  NGJ,  CIRCM,  IFPC.  In  the  case  of  NGJ  and  CIRCM,  that  seems 
inconsistent  with  the  need  to  counter  near-tenn,  or  even  current,  threats. 

•  Process  delays,  such  as  protests,  acquisition  strategy  reformulation,  and  contract  rework  have 
impacts,  measured  more  in  months  than  in  years — but  they  do  not  appear  to  be  “long  poles  in 
tent”  relative  to  other  factors. 

-  A  possible  exception  is  3DELRR.  This  program  is  currently  the  subject  of  a  lawsuit 
regarding  the  resolution  of  protests  of  the  EMD  contract  award.  The  program  will 
remain  on  hold  until  the  suit  it  resolved — an  indetenninate  period  of  time. 

Broader  Insights  on  the  Importance  of  Cycle  Time 

Our  research  has  led  to  some  broader  thoughts  that  fall  under  the  rubric  of  “cycle  time.” 

While  much  has  been  said  and  written  on  defense  acquisition  cycle  time,  often  these  thoughts 
do  not  include  the  obvious  question:  Why  is  there  concern  over  acquisition  cycle  time?  In  our  view, 
reducing  cycle  time  per  se  should  not  be  the  prime  objective,  since  that  may  lead  to  a  number  of 
undesired  outcomes,  such  as  greatly  increased  costs  and/or  inadequate  perfonnance.  At  times  it  may 
be  impossible  to  achieve  stated  performance  requirements  faster  given  technological  limitations. 
Trading  off  the  “required”  performance  to  get  something  out  quickly  has  attendant  downsides — 
notably  the  inability  to  meet  what  users  say  they  need.  Rather,  the  cycle-time  goal  should  be  to 
structure  and  execute  acquisition  programs  to  meet  the  needs  of  operational  forces,  in  terms  of  both 
capabilities  and  timeliness.  Though  simply  stated,  achieving  that  end  has  many  dimensions.  It  may 
not  be  possible  to  provide  the  capabilities  desired  by  the  user  within  the  desired  time  constraint — in 
fact,  in  today’s  world  of  rapid  technological  dispersion,  that  situation  might  be  the  norm.  So  tradeoffs 
must  be  made  to  provide  the  best  capabilities  achievable  by  the  time  needed  within  an  acceptable 
level  of  risk. 

Thus,  the  real  “cycle  time”  issue  is  how  to  provide  the  operational  forces  weapon  systems  that 
have  the  needed  capabilities  and  that  are  fielded  when  needed.  To  ensure  that  those  dual  objectives 
are  met,  requirements  and  acquisition  communities  must  work  together  closely  in  the  initial  stages  of 
systems  acquisition  to  define  programs  based  on  technologies  that  can  be  matured  and  implemented 
in  the  required  timeframe  within  an  acceptable  level  of  risk.  If  that  process  is  not  effectively 
accomplished,  the  ensuing  program  faces  a  high  risk  of  failure  to  achieve  its  fundamental  objective 
of  providing  the  user  the  right  capabilities  at  the  right  time. 

There  is  no  “one  size  fits  all”  approach  to  achieving  the  fundamental  objective.  At  one 
extreme,  there  are  rare  situations  where  it  is  necessary  to  seek  highly  ambitious  capabilities,  to  accept 
higher  risks,  to  confront  unknowns.  Schedule  and  cost  estimates  will  be  more  uncertain.  It  is  the  role 
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of  leadership  to  determine  whether  the  requirements  for  such  capabilities  and  their  operational  value 
justify  taking  on  higher  risks.  The  risks  must  be  identified  and  articulated  carefully  to  leadership  so 
that  an  informed  decision  can  be  reached.32 

At  the  other  extreme  are  the  “rapid  acquisition”  programs  that  meet  immediate  needs  and 
therefore  should  be  acquired  using  a  low-risk,  predictable  approach  with  well-defined,  currently 
available  technologies.  Those  programs  are  usually  not  MDAPs,  though  there  have  been  exceptions 
(notably  the  Stryker  and  MRAP  programs). 

Most  MDAPs  fall  between  these  extremes,  and  for  most  of  them  getting  it  right  is  more 
important  than  getting  it  soon.  Undue  emphasis  on  cycle  time  and  predictability  of  schedule  can 
inhibit  pursuit  of  programs  that  are  aimed  at  difficult  objectives  that  may  take  more  time  to  achieve. 
However,  that  does  not  mean  being  complacent  in  requiring  that  programs  that  aim  high;  be  well- 
founded  on  clear  assessment  of  underlying  assumptions  regarding  technical  risk;  and  have  a  well- 
formulated  program  strategy  to  address  that  risk.  If  a  program  is  reaching  for  performance  that  stresses 
the  state  of  art,  then: 

•  The  rationale  in  tenns  of  future  threat  or  compelling  operational  need  should  be  clearly 
identified; 

•  Risks  should  be  explicitly  defined,  independently  verified,  and  made  clear  to  all 
management  levels; 

-  There  should  be  well-defined  approaches  pursued  to  reduce  the  risk  by  carefully 
maturing  the  technologies  and  addressing  their  integration  into  the  system  as  the 
system  evolves  over  time, 

—  Independent  assessments  of  risks  should  be  the  job  of  the  systems  engineering 
establishments  (OSD  and  Services); 

•  The  means  for  accommodating  technology  improvement  and  change  into  the  system  as  it 
develops  and  is  fielded  overtime  should  be  a  clear  part  of  the  acquisition  strategy;  and 

•  The  program  should  receive  high-level  oversight  throughout.33 

Frequently,  the  acquisition  system  must  deal  with  problems  created  by  inadequately  thought- 
through  program  initiation,  attempting  to  achieve  high-impact  defense  capabilities  on  an  unexecutable 
schedule.  Clearly  this  is  a  problem  that  DOD  needs  to  address  more  broadly  and  strategically. 


32 

~  A  good  depiction  of  approaches  to  address  technology  level  and  time  to  field  is  shown  in  Army  Strong:  Equipped, 
Trained  and  Ready.  Final  Report  of  the  2010  Army  Acquisition  Review,  January  2011,  Figure  41,  p.  100. 

33 

The  FI  17  stealth  aircraft  is  a  well-known  example  of  this  approach.  See  Richard  Van  Atta,  et  ah.  Transformation  and 
Transition:  DARPA ’s  Role  in  Fostering  an  Emerging  Revolution  in  Military  Affairs,  Volume  1,  IDA  Paper  P-3698, 
Alexandria,  VA:  Institute  for  Defense  Analyses,  April  2003,  pp.  1 1-15. 
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Addressing  it  requires  rigorous  assessments  of  the  state  of  technologies,  informed  by  technology 
prototyping,  experimentation,  and  demonstration  of  advanced  concepts,  to  assess  the  prospects  and 
potential  value  to  the  user.  The  maturation  of  underlying  enabling  technologies  needs  to  be  given 
adequate  and  sufficiently  early  focus  and  funding.  Without  such  preliminary  steps,  the  risk  will  fall 
onto  the  MDAPs,  which  then  proceed  with  too  little  information  and  too  little  certainty  and  thus  run 
risks  of  unachievable  schedules,  cost  growth,  or  even  program  cancellation. 
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Appendix  A 

Requirements  Documents  Identified/Reviewed 


IDA 


Requirements  Documentation 


Program 

CBA 

ICD 

CDD/CPD 

AoA 

NGJ 

IOC:  3QFY21 

EW  CBA 

2009a 

EW  ICD  2009a 

Feb. '15  draft 

July  2011 

AMDR 

IOC:  4QFY23 

Integrated  Air 
&  Msl.  Def. 
'09. 

MAMDJFb  ICD 

2006 

Radar/Hull  Study 
2009 

Apr.  2013 

MAMDJF 

Aug.  2007 

CIRCM 

IOC:  4QFY20 

JCB  "Mini- 
CBA"a 

AS  FSA  2008 

Aircraft 

Survivability  (AS) 
2011a 

Oct.  2013 

Waived  by 

DAE  per  Army 
request 

IFPC 

FUE:  4QFY19 
IOC:  3QFY20? 

None 

identified 

Int.  Unit,  Base 
Install.  Protection 

2009 

Air  &  Msl  Def 
SysofSys  2010a 

Apr.  2013 
(revised)3 

3DELRR 

IOC:  2QFY20 

None 

identified 

June  2012a 

July  2013  CPD 

2006  CDD  by 
USMC 

Apr.  2014a 

F-22  3.2B  Mod 
RAA:  4QFY19 

None 

identified 

None  identified 

Enhanced  Global 

Strike  2007a 

Incr.  3.2B  KSAs 
Sep.2012a 

Apr.  2011a 

a  Not  obtained 

b  Maritime  Air  &  Missile  Defense  of  the  Joint  Force 
c  The  Functional  Solutions  Analysis  and  Functional  Needs  Analysis 
(CBA  components)  are  appendices  to  the  I  CD 


Figure  11.  Requirements  Documentation  for  MDAPs  Investigated 
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Appendix  B 

Provisions  Regarding  Establishment  of  Schedules  in 
DOD  and  Joint  Staff  Issuances 


The  study  team  reviewed  both  Joint  Capabilities  Integration  and  Development  System 
(JCIDS)  documentation  and  Department  of  Defense  Instruction  (DODI)  5000.02  for 
specifications  regarding  schedules  and  schedule  setting. 

JICDS  documentation 

Reviewed  CJCSI34  3170.011  (Jan.  20150)  Joint  Capabilities  Integration  and  Development 
System  (JCIDS)  and  Joint  Staff  Manual  for  the  Operation  of  the  Joint  Capabilities  Integration 
and  Development  System  (JCIDS) 

CJCSI  3170.011  contains  no  explicit  references  at  all  to  Initial  Operational  Capability 
(IOC)  or  schedule  setting.  It  does  have  one  very  general  provisions  regarding  performance,  cost, 
and  schedule  tradeoffs,  but  nothing  addressing  schedule  tradeoffs  per  se — e.g.,  whether  it’s  better 
to  have  lesser  capabilities  sooner  or  greater  capabilities  later. 

The  associated  manual,  on  the  other  hand,  contains  numerous  references;  which  are 
summarized  below: 

•  Capabilities  Based  Assessments  (CBAs)  are  to  “consider  the  timeframe  under 
consideration,  applicable  threats,  ...joint  [operational]  concepts,  and  related  effects  to  be 
achieved.”  The  timeframe  is  “important  both  to  help  establish  the  conditions  and  threats 
under  which  the  mission  is  to  be  carried  out,  and  as  a  key  component  in  discussions 
between  the  requirement  Sponsor  and  the  acquisition  community  in  detennining  the 
required  IOC  and  FOC  dates.” 

•  Initial  Capabilities  Documents  (ICDs)  are  to  “Identify  the  timeframe  under  consideration 
for  IOC  and  Full  Operational  Capability  (FOC)  based  on  input  from  supported/supporting 
Combatant  Commands  and  the  acquisition  community.” 

•  Capabilities  Development  Documents  (CDDs)  are  to  “Define  what  actions,  when 
complete,  will  constitute  attainment  of  IOC  and  FOC  of  the  current  increment.  Specify 
the  target  date  for  IOC  and  FOC  attainment  based  on  discussions  and  coordination 
between  the  requirement  Sponsor  and  the  acquisition  community.  Describe  the  types  and 


34  Chairman,  Joint  Chiefs  of  Staff  Instruction. 
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quantities  of  assets  required  to  attain  IOC  and  FOC.”  (An  identical  provision  is  included 
for  Capabilities  Production  Documents  (CPDs).) 

It  also  has  provisions  regarding  tradeoffs  among  performance,  cost,  and  schedule. 

DODI  5000.02,  Operation  of  the  Defense  Acquisition  System  (Jan.  2015). 

DODI  5000.02  also  has  no  explicit  provision  regarding  the  “less  capable  sooner  vs.  more 
capable  later”  question. 

It  does,  however,  have  several  provisions  regarding  tradeoffs  between  performance,  cost, 
and  schedule. 

The  first  is  within  the  systems  engineering  context  and  what  is  called  “Development 
Planning”:35 

In  preparation  for  the  Materiel  Development  Decision,  and  to  inform  an  Analysis  of 
Alternatives  (AoA),  the  DOD  Components  will  conduct  early  systems  engineering 
analyses  and  conduct  an  assessment  of  how  the  proposed  candidate  materiel  solution 
approaches  are  technically  feasible  and  have  the  potential  to  effectively  address 
capability  gaps,  desired  operational  attributes,  and  associated  external  dependencies. 

During  the  acquisition  life  cycle,  the  Program  Manager  will  conduct  systems 
engineering  trade-off  analyses  to  assess  system  affordability  and  technical  feasibility 
to  support  requirements,  investment,  and  acquisition  decisions.  Systems  engineering 
trade-off  analyses  will  depict  the  relationships  between  system  life-cycle  cost  and  the 
system’s  performance  requirements,  design  parameters,  and  delivery  schedules. 

In  support  of  the  validation  of  the  Capability  Development  Document  (or  equivalent 
requirements  document),  the  Program  Manager  will  conduct  a  systems  engineering 
trade-off  analysis  showing  how  cost  varies  as  a  function  of  system  requirements 
(including  Key  Perfonnance  Parameters),  major  design  parameters,  and  schedule. 

The  results  will  be  provided  to  the  MDA  and  will  identify  major  affordability  drivers 
and  show  how  the  program  meets  affordability  constraints 

How  these  results  are  “provided  to  the  MDA”  is  not  known.  Neither  the  Service  staffs  nor 
the  program  offices  contacted  infonned  the  study  team  of  any  such  tradeoffs  despite  being 
specifically  asked. 

The  next  discussion  of  tradeoffs  is  under  the  Analysis  of  Alternatives  (AoA)36 
procedures: 

2.  AOA  PROCEDURES 


35  DODI  5000.02,  JANUARY  2015,  Enel. 3,  para.  2,  (page  82). 

36  DODI  5000.02  January  2015,  Enel.  9,  para.  2,  (page  125). 
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a.  The  Director  of  Cost  Assessment  and  Program  Evaluation  (DCAPE)  develops  and 
approves  study  guidance  for  the  AoA  for  potential  and  designated  Acquisition 
Category  I  and  IA  programs  and  for  each  joint  military  or  business  requirement  for 
which  the  Chairman  of  the  Joint  Requirements  Oversight  Council  (JROC)  or  the 
Investment  Review  Board  is  the  validation  authority.  In  developing  the  guidance,  the 
DCAPE  solicits  the  advice  of  other  DOD  officials  and  ensures  that  the  guidance 
requires,  at  a  minimum: 

(1)  Full  consideration  of  possible  tradeoffs  among  life-cycle  cost,  schedule,  and 
perfonnance  objectives  (including  mandatory  key  perfonnance  parameters)  for  each 
alternative  considered. 

(2)  An  assessment  of  whether  the  joint  military  requirement  can  be  met  in  a  manner 
consistent  with  the  cost  and  schedule  objectives  recommended  by  the  JROC  or  other 
requirements  validation  authority. 

(3)  Consideration  of  affordability  analysis  results  and  affordability  goals  if 
established  by  the  Milestone  Decision  Authority  (MDA). 

Once  the  AoA  is  completed,  DCAPE  performs  an  assessment  and  reports  the  results  in  a 
memorandum  to  the  MDA:37 

In  the  memorandum,  the  DCAPE  assesses: 

(1)  The  extent  to  which  the  AoA: 

(a)  Examines  sufficient  feasible  alternatives 

(b)  Considers  tradeoffs  among  cost,  schedule,  sustainment,  and  required  capabilities 
for  each  alternative  considered 

(c)  Achieves  the  affordability  goals  established  at  the  MDD  and  with  what  risks 

(d)  Uses  sound  methodology 

The  last  area  in  which  tradeoffs  are  mentioned  in  DODI  5000.02  is  in  the  Developmental 
Test  and  Evaluation  (DT&E)  context:38 

DT&E  ACTIVITIES 

a.  DT&E  activities  will  start  when  requirements  are  being  developed  to  ensure  that 
key  technical  requirements  are  measurable,  testable,  and  achievable. 

b.  A  robust  DT&E  program  includes  a  number  of  key  activities  to  provide  the  data 
and  assessments  for  decision  making.  The  DT&E  program  will: 

(1)  Verify  achievement  of  critical  technical  parameters  and  the  ability  to  achieve  key 
perfonnance  parameters,  and  assess  progress  toward  achievement  of  critical 
operational  issues 


37  DODI  5000.02  January  2015,  Enel.  9,  para.  2(c),  (page  126). 

38  DODI  5000.02  January  2015,  Enel.  4,  para.  4,  (page  91). 
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(2)  Assess  the  system’s  ability  to  achieve  the  thresholds  prescribed  in  the  capabilities 

documents 

(3)  Provide  data  to  the  Program  Manager  to  enable  root  cause  determination  and  to 

identify  corrective  actions 

(4)  Validate  system  functionality 

(5)  Provide  infonnation  for  cost,  perfonnance,  and  schedule  tradeoffs 

(6)  Assess  system  specification  compliance 

Note  that  in  all  these  instances,  while  schedule  is  mentioned  as  a  tradeoff  parameter,  it 
receives  no  particular  emphasis  (unlike  affordability).  There  is  no  provision  directing  explicit 
consideration  of  whether  capabilities  can  be  provided  in  time  to  meet  operational  needs  nor  the 
tradeoffs  between  employing  less  advanced  (and  less  risky)  technologies  to  achieve  a  high 
confidence  of  meeting  a  need  versus  more  advanced  technologies  providing  greater  capabilities 
later  (but  probably  with  higher  risks). 

Virtually  all  of  these  provisions  are  new  for  this  issuance  of  5000.02,  which  first  appeared 
in  “interim”  fonn  in  November  2013.  Since  all  the  programs  investigated  in  depth  in  this  task  had 
MDDs  or  other  initial  milestone  decisions  prior  to  that  date,  they  were  all  fonnulated  based  on  the 
2008  version  of  DODI  5000.02.  Thus  they  arguably  cannot  be  held  accountable  for  not  doing  the 
tradeoff  analyses  called  for  in  the  above  excerpts.  However,  Directive-Type  Memorandum  10- 
017,  “Development  Planning  to  Inform  Materiel  Development  Decision  (MDD)  Reviews  and 
Support  Analyses  of  Alternatives  (AoA),”  covering  the  development  planning  provisions  was 
issued  originally  in  September  2010. 

The  earlier  editions  of  CJCS  3170  and  the  accompanying  manual  were  not  significantly  different 
from  the  current  version  with  regard  to  scheduling  issues. 
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Appendix  D  Abbreviations 


3DELRR 

ATIRCM 

AMDR 

AoA 

CBA 

CMWS 

CDD 

CIRCM 

CM 

CPD 

DCAPE 

DOD 

DODI 

EMD 

FOC 

FY 

GaN 

ICD 

IFPC 

IDA 

IOC 

IOT&E 

IR 

JCIDS 

JROC 

KPP 

MDA 

MDAP 

MDD 

MRAP 


3 -Dimensional  Expeditionary  Long-Range  Radar 
Advanced  Threat  Infrared  Countenneasure 
Air  and  Missile  Defense  Radar 
Analysis  of  Alternatives 
Capabilities  Based  Assessment 
Common  Missile  Warning  System 
Capabilities  Development  Document 
Common  Infrared  Countermeasures 
Cruise  Missile 

Capabilities  Production  Document 

Director,  Cost  Analysis  and  Program  Evaluation 

Department  of  Defense 

Department  of  Defense  Instruction 

Engineering  and  Manufacturing  Development 

Full  Operational  Capability 

Fiscal  Year 

Gallium  Nitride 

Initial  Capabilities  Document 

Integrated  Force  Protection  Capability 

Institute  for  Defense  Analyses 

Initial  Operational  Capability 

Initial  Operational  Testing  and  Evaluation 

Infrared 

Joint  Capabilities  Integration  and  Development 
System 

Joint  Requirements  Oversight  Council 
Key  Perfonnance  Parameter 
Milestone  Decision  Authority 
Major  Defense  Acquisition  Program 
Materiel  Development  Decision 
Mine-Resistant  Ambush  Protected 
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NGJ 

Next  Generation  Jammer 

OUSD(AT&L) 

Office  of  Under  Secretary  of  Defense,  Acquisition 

OSD 

Office  of  Secretary  of  Defense 

RAM 

Rockets,  Artillery  and  Mortar 

RDT&E 

Research  Development  Test  and  Evaluation 

SARs 

Selected  Acquisition  Reports 

SWAP 

Space,  Weight,  and  Power 

TMRR 

Technology  Maturation  and  Risk  Reduction 

UAV 

Unmanned  Aerial  Vehicle 

USD 

Under  Secretary  of  Defense 

use 

United  States  Code 
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